ASO xdate error metrics
Dominik Schneider
- r2
- % MAE
- are the differences in patterns of fsca and swe similar? can fsca patterns be used to pick the date to predict swe?
- compare prediction errors vs basin aggregated snow variables
- compare prediction errors vs fsca pattern differences
- compare best pred vs lowestfscamae pred
- swe and fsca difference for best predictions
r2
| mdlpreds | mean(r2) | median(r2) | sd(r2) |
|---|---|---|---|
| phv | 0.52 | 0.56 | 0.09 |
| phvfsca | 0.76 | 0.77 | 0.06 |
| phvaso | 0.67 | 0.71 | 0.17 |
| phvasofsca | 0.79 | 0.79 | 0.09 |
phv
phvfsca
phvaso
phvasofsca
all model boxplots
% MAE
| mdlpreds | mean(pctmae) | median(pctmae) | sd(pctmae) |
|---|---|---|---|
| phv | 59.38 | 54.39 | 21.96 |
| phvfsca | 37.14 | 37.61 | 11.93 |
| phvaso | 53.98 | 45.90 | 23.32 |
| phvasofsca | 36.21 | 35.70 | 12.32 |
phv
phvfsca
phvaso
phvasofsca
all model boxplots
are the differences in patterns of fsca and swe similar? can fsca patterns be used to pick the date to predict swe?
comparison of maps for each date but leaving out maps from the same year in the comparison
with mean absolute error
with mean squared error
more dates are statistically significant with mae so we’ll stick with that.
compare prediction errors vs basin aggregated snow variables
compare with best errors
this doesn’t show much. errors go up end of season while mean swe goes down….
compare with all errors
compare prediction errors vs fsca pattern differences
compare with best errors only
again not a very useful plot. errors go up as fsca mae goes down (due to smaller differences when there is less snow cover)
compare with all errors by date
see “figs/so_many_fscaplots.pdf” for all plots
Table of significant slopes
| simdte | phv | phvfsca | phvaso | phvasofsca | sep | simdte.1 | phv.1 | phvfsca.1 | phvaso.1 | phvasofsca.1 |
| 2013-04-03 | -0.24 | -0.07 | -0.26 | - | | | 2013-04-03 | 87.25 | - | 113.44 | - |
| 2013-04-29 | -0.24 | - | -0.32 | -0.17 | | | 2013-04-29 | 73.76 | - | 100.15 | - |
| 2013-05-03 | -0.19 | - | - | - | | | 2013-05-03 | 60.13 | - | 67.53 | - |
| 2013-05-25 | - | -0.09 | -0.18 | -0.18 | | | 2013-05-25 | 163.89 | - | 213.94 | - |
| 2013-06-01 | -0.09 | -0.15 | -0.16 | -0.24 | | | 2013-06-01 | 285.64 | - | 360.21 | - |
| 2013-06-08 | -0.13 | -0.25 | -0.19 | -0.37 | | | 2013-06-08 | 618.97 | 157.38 | 762.56 | - |
| 2014-03-23 | -0.20 | - | -0.16 | - | | | 2014-03-23 | 74.09 | -52.27 | 97.27 | -51.56 |
| 2014-04-07 | -0.20 | - | -0.17 | - | | | 2014-04-07 | 72.32 | -54.53 | 92.60 | -52.38 |
| 2014-04-13 | -0.15 | - | -0.12 | - | | | 2014-04-13 | 56.55 | -65.95 | 87.28 | -54.56 |
| 2014-04-20 | - | - | - | - | | | 2014-04-20 | - | - | 93.80 | - |
| 2014-04-28 | - | - | - | -0.20 | | | 2014-04-28 | - | - | 81.06 | - |
| 2014-05-02 | - | - | - | - | | | 2014-05-02 | - | - | - | - |
| 2014-05-11 | - | - | - | - | | | 2014-05-11 | - | - | - | - |
| 2014-05-17 | -0.28 | -0.20 | -0.41 | -0.44 | | | 2014-05-17 | - | - | - | - |
| 2014-05-27 | -0.29 | -0.28 | -0.42 | -0.49 | | | 2014-05-27 | 499.44 | - | 670.84 | - |
| 2014-05-31 | -0.27 | -0.30 | -0.37 | -0.54 | | | 2014-05-31 | 730.34 | - | 935.01 | - |
| 2015-02-17 | -0.54 | -0.33 | -0.50 | - | | | 2015-02-17 | - | - | - | - |
| 2015-03-05 | -0.29 | -0.11 | -0.32 | -0.17 | | | 2015-03-05 | - | - | - | - |
| 2015-03-25 | - | - | - | 0.08 | | | 2015-03-25 | - | - | 292.38 | - |
| 2015-04-09 | -0.16 | -0.19 | -0.22 | -0.24 | | | 2015-04-09 | - | - | - | - |
| 2015-04-15 | - | - | - | - | | | 2015-04-15 | 254.24 | - | 358.00 | - |
| 2015-04-27 | -0.28 | -0.10 | -0.21 | - | | | 2015-04-27 | - | - | - | - |
| 2015-05-01 | - | - | -0.07 | -0.12 | | | 2015-05-01 | 546.20 | - | 735.10 | - |
| 2015-06-08 | -0.14 | -0.23 | -0.19 | -0.57 | | | 2015-06-08 | 1298.93 | 377.43 | 1466.82 | 543.49 |
| 2016-05-27 | -0.12 | -0.12 | - | -0.15 | | | 2016-05-27 | 73.24 | - | 86.43 | - |
| 2016-06-07 | -0.19 | - | - | - | | | 2016-06-07 | 54.77 | - | 67.80 | - |
| 2016-06-13 | -0.16 | - | -0.18 | -0.16 | | | 2016-06-13 | 109.99 | - | 158.43 | - |
| 2016-06-20 | -0.11 | -0.15 | -0.17 | -0.30 | | | 2016-06-20 | 183.25 | 28.92 | 261.79 | 19.85 |
combine pred error best fit lines by mdlpreds, plot mean +/- se
compare best pred vs lowestfscamae pred
| mdlpreds | var | mean(val) | sd(val) |
|---|---|---|---|
| phv | diff | -0.03 | 0.02 |
| phvaso | diff | -0.04 | 0.04 |
| phvasofsca | diff | -0.04 | 0.06 |
| phvfsca | diff | -0.02 | 0.02 |
| errorvar | mdlpreds | var | mean(val, na.rm = T) | sd(val, na.rm = T) |
|---|---|---|---|---|
| r2 | phv | bestpred | 0.52 | 0.09 |
| r2 | phv | lowestfscamae | 0.50 | 0.10 |
| r2 | phvaso | bestpred | 0.67 | 0.17 |
| r2 | phvaso | lowestfscamae | 0.63 | 0.18 |
| r2 | phvasofsca | bestpred | 0.79 | 0.09 |
| r2 | phvasofsca | lowestfscamae | 0.75 | 0.10 |
| r2 | phvfsca | bestpred | 0.76 | 0.06 |
| r2 | phvfsca | lowestfscamae | 0.74 | 0.06 |
| mdlpreds | var | mean(val) | sd(val) |
|---|---|---|---|
| phv | diff | 22.70 | 38.00 |
| phvaso | diff | 23.83 | 34.42 |
| phvasofsca | diff | 30.42 | 29.96 |
| phvfsca | diff | 29.93 | 40.43 |
| errorvar | mdlpreds | var | mean(val, na.rm = T) | sd(val, na.rm = T) |
|---|---|---|---|---|
| pctmae | phv | bestpred | 59.38 | 21.96 |
| pctmae | phv | lowestfscamae | 82.08 | 39.06 |
| pctmae | phvaso | bestpred | 53.98 | 23.32 |
| pctmae | phvaso | lowestfscamae | 77.81 | 37.47 |
| pctmae | phvasofsca | bestpred | 36.21 | 12.32 |
| pctmae | phvasofsca | lowestfscamae | 66.63 | 27.49 |
| pctmae | phvfsca | bestpred | 37.14 | 11.93 |
| pctmae | phvfsca | lowestfscamae | 67.06 | 39.15 |
swe and fsca difference for best predictions
More yellow means prediction was better. x and y axis represent the differences between simulation date and model date expressed as mean squared error or mean absolute error.
Again not that useful to look at because prediction error is lowest when fscamae is highest (due to more snow cover earlier in season). Not immediately intuitive until you look at fscamae vs date.